Human Tracking Using Delphi ESR-Vision Fusion in Complex Environments

A variety of UGV (Unmanned Ground Vehicle) applications pose the challenge that UGVs need to handle human detection and tracking in complex environments that include dusty, smoky and foggy conditions. These environments make a vision-based human tracking-by-detection system ineffective. To cope with this challenge, we build a radar-vision fusion system, utilizing a 77GHz 2D Delphi ESR (Electronically Scanning Radar) and a CCD camera. Our fusion system utilizes radar returns to generate ROIs (Regions of Interests) and then employs a vision-based human detection technique to validate ROIs. Considering that human are weak targets for a 77GHz radar sensor due to their smaller sizes and weaker reflectivity, we develop a human tracking approach to recover from intermittent human misses. This improves the accuracy of our multi-sensor system. We design experiments to study the behavior of Delphi ESR for human detection. We also characterize Delphi ESR’s measurement noise. Using the derived Gaussian noise model parameters, we develop a novel human tracking approach using Kalman filter. We also describe, in detail, an approach to map radar returns to image plane for generating ROIs. A set of real-world experiments show the effectiveness of our approach in human tracking and radar-vision registration.